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Background/Context: Underprepared students at community colleges are often assigned to a sequence of developmental education 
courses that can substantially delay, or even halt, their progress to degree completion. In 2014, Florida implemented a comprehensive 
reform under Senate Bill (SB) 1720 that allowed the majority of incoming students to enroll directly into college-level courses, while 
remaining developmental education courses were offered in new instructional modalities that were designed to be completed more quickly 
than traditional semester-long courses. Colleges provided extensive advising and academic support services intended to help students 
succeed while progressing at a more accelerated pace. 


Purpose/Objective/Research Question/Focus of Study: In this study, we examined the impact of Florida's developmental education 
reform on early momentum. More specifically, we defined early momentum as student success in early outcomes (such as the number of 
credits attempted and earned), which may set students on a promising long-term trajectory on subsequent postsecondary outcomes. 


Population/Participants/Subjects: Our sample included all first-time-in-college students enrolled in all 28 public state colleges. We 
included three cohorts of students before the reform and up to three cohorts of students after the reform. Each cohort consisted of 
approximately 70,000 students. 


Research Design: We used an interrupted time series design to compare student outcomes three years before the reform with those up to 
three years after the reform. Our outcome variables, measured one and three years following initial college enrollment, represented 
continuous indicators for the number of college-level credits attempted and the number of college-level credits earned. We also examined 
whether the data revealed heterogeneity in the reform’s impacts by race/ethnicity, family income status, and level of high school academic 
preparation. 


Findings/Results: We found small positive effects on all outcomes, indicating that the reform accelerated student success in both the short 
term and longer term. The impacts of the reform were even greater for Black, Hispanic, Indigenous, low-income, and underprepared 
students (particularly in the first year), thus reducing existing achievement gaps. 


Conclusions/Recommendations: Florida’ SB 1720 consisted of a complementary set of reform efforts that may together have a larger 
impact than any single component alone. The results suggest that initial momentum gains in the first year may have set some students on 
a more successful long-term trajectory, particularly those most likely to be assigned to developmental education before the reform. 
Colleges should continue to provide comprehensive student support services to help students succeed while progressing at a more 
accelerated pace. 


While community colleges seek to provide open access to students regardless of their academic preparation, many incoming students face 
barriers to enrollment in college-level courses because of institutional requirements for placement testing and developmental education 
courses. Nationwide, approximately 41% of students entering public two-year colleges reported enrolling in at least one developmental 
education course, with even higher rates of enrollment among Black, Hispanic, low-income, and first-generation students (X. Chen, 
2016). Developmental education courses—courses that are not college level but are often required as prerequisites for underprepared 
students—can add numerous courses to students’ degree plans because there is often a sequence consisting of multiple levels of 
developmental education courses based on students’ test score performance. Further, students may face requirements to enroll in 
developmental education courses in multiple subject areas of reading, writing, and math. 


Specifically, developmental education requirements may hinder students’ academic progress in two ways. First, the extra course 
requirements may result in additional points in time at which students may stop out. Bailey et al. (2010) found that among students 
assigned to developmental education, only about half complete all the courses required in the sequence. Most of the students did not 
complete the course sequence, not because they failed a developmental education course, but because they did not return to take the next 
course in the sequence. Second, even among students who successfully complete the developmental education sequence, the additional 
course requirements tend to lengthen the time to degree completion. Progression through the developmental sequence also varies by 
student characteristics; part-time students, males, older students, African Americans, and vocational students are less likely to complete 
all their developmental requirements. Bailey and colleagues (2010) concluded that 


if there is a national goal to increase college success and graduation rates, that increase is going to have to come from among these 
types of students. The goal of educators therefore must be to try to lower the cost in time and resources to the student of 


successfully navigating the developmental sequence. (p. 268) 


More than a half dozen states have attempted to reduce developmental education requirements by implementing reforms to placement 
policies or adopting new instructional modalities for developmental education courses (Scott-Clayton, 2018). College placement tests 
tend to disproportionately underplace students into developmental education courses when many of these students likely could have 
passed college-level courses (Leeds & Mokher, 2019; Ngo & Melguizo 2015; Scott-Clayton et al., 2014). States such as Minnesota, 
Oregon, and Washington have sought to reduce these misplacement rates by either requiring or encouraging colleges to consider 
additional indicators of high school academic achievement in placement decisions. Other states, such as California, Tennessee, and Texas, 
have implemented new instructional modalities intended to help students progress more quickly through developmental education, such 
as corequisite developmental education courses that students take in the same semester as college-level courses. 


In 2014, Florida’s Senate Bill (SB) 1720 took a comprehensive approach to developmental education, which required all Florida College 
System (FCS) institutions to make several significant changes simultaneously. First, recent high school graduates and active duty military 
personnel became exempt from placement testing and had the option to enroll directly in college-level courses regardless of their prior 
academic preparation. Second, colleges had to replace traditional semester-long developmental education courses with new instructional 
strategies that would allow students to accelerate their progression to college-level courses. These include corequisite, contextualized, 
modularized, and compressed courses, which we describe further in the next section. Third, the policy mandated colleges to offer 
enhanced advising and support services to help students make informed decisions about course placement and provide additional support 
for those struggling academically in developmental and college-level courses. Although SB 1720 was not specifically designed as an 
equity-focused reform, students of color and low-income students have historically been more likely to be placed into developmental 
education, so they have the potential to be more strongly affected by the reform’s impacts. 


The component of Florida’s reform that made developmental education optional for the majority of incoming students led to controversy 
among both researchers and practitioners. An article in the National Education Association’s publication, neaToday, called Florida’s 
legislation “another lousy ‘reform’ idea” that would harm students considered academically unprepared for college-level work (Flannery, 
2014). Tierney and Duncheon (2013) echoed this concern, noting that although eliminating developmental education could help some 
students who are close to college-ready, failure rates would likely increase among lower performing students. Critics also expressed 
concerms that developmental education reforms may have particularly harmful effects on disadvantaged subgroups such as low-income, 
underrepresented minority, nontraditional, and first-generation students, thus further exacerbating existing achievement gaps in 
postsecondary outcomes such as degree completion (Squires & Boatman, 2014). 


In this study, we examine the impact of Florida’s developmental education reform on early momentum. More specifically, we defined 
early momentum as student success in early outcomes (such as the number of credits attempted and earned) that may set students on a 
promising long-term trajectory on subsequent postsecondary outcomes. According to a recent study by Belfield et al. (2019), first-year 
measures of college-level credit completion are key predictors of longer term student success and may be particularly important for 
explaining racial equity gaps in college completion rates. Using student-level records from the population of first-time-in-college (FTIC) 
students at all 28 FCS institutions, we employ an interrupted time series (ITS) design to compare student outcomes three years before the 
reform with those up to three years after the reform. We also examine whether the data reveal heterogeneity in the reform’s impacts by 
race/ethnicity, family income status, and level of high school academic preparation. Specifically, we address the following research 
questions: (1) How has the number of college-level credits attempted and earned in the first and third years of enrollment changed 
following the developmental education reform among FTIC students enrolled in the FCS institutions? (2) Did the impact of the reform 
differ by racial/ethnic background, free or reduced-priced lunch (FRL) status, or level of high school academic preparation? 


This study makes several unique contributions to inform both research and practice. First, prior studies have primarily looked at the 
impact of individual components of developmental education reform, such as the use of multiple measures for placement or the 
implementation of corequisite developmental courses. Our study examines the impact of a complementary set of reform efforts that may 
together have a larger impact than any single component alone. For example, if developmental education becomes optional, it may be 
particularly important to provide additional academic support services to students who opt out. Second, in addition to examining whether 
the reform had the intended effect of accelerating student success in higher education, we also explored whether unintended consequences 
may have resulted for low-income students or those from diverse racial/ethnic backgrounds, including Asian, Black, Hispanic, 
Indigenous, and multiracial students. This was an initial concern among critics of Florida’s policy, and our analyses provide insight into 
whether these concerns were realized. Our study also expands prior work by going beyond exploring Black—White and Hispanic—White 
achievement gaps, to examining differential impacts among a broad range of racial/ethnic groups, including Indigenous students—a 
population often excluded from studies of higher education (Shotton et al., 2013). Third, most rigorous evaluations of developmental 
education use regression discontinuity designs to estimate impacts for students who score close to the college-ready cutoff, but some 
studies that include a broader sample of students indicate that program impacts may vary based on students’ academic performance (e.g., 
Boatman & Long, 2018). Our subgroup analyses explore whether Florida’s reform had differential impacts for underprepared students 
who completed only a basic sequence of math or English courses in high school. Taken together, the findings from this study can inform 
our understanding of how changes to developmental education differentially affect student subgroups and may help policy makers better 
design reform efforts to support the success of students from a broad range of backgrounds. 


In the next section, we describe previous literature examining the impact of developmental education on student outcomes such as credit 
accumulation, and we provide evidence from prior studies to support our hypotheses of how Florida’s developmental education reform is 
intended to improve these outcomes for underprepared students. We then describe our methods using an ITS approach, and the data that 
we used, which consisted of longitudinal student records from Florida’s 28 public FCS institutions. We present our findings and conclude 
with a discussion of the implications for both research and policy. 


LITERATURE REVIEW 


A large body of empirical evidence has examined the impact of assignment to developmental education on students’ academic progress, 
particularly among students on the margins of college readiness. Valentine et al. (2017) conducted a meta-analysis of regression 
discontinuity studies examining the impact of assignment to developmental education on the likelihood of passing college-level courses, 
credit accumulation, and degree completion. The results indicate that students assigned to developmental education who score just below 
college-ready perform significantly worse on all three outcomes relative to those assigned to college-level courses, particularly in reading 
and writing. Among the 16 studies in the meta-analysis that examined credit accumulation, students narrowly assigned to developmental 
education completed an average of about two to three fewer credits after three years. 


Students from underrepresented racial and ethnic groups may face even greater challenges in developmental education because of issues 
of discrimination and racism, structural inequalities, and a failure of institutions to support their unique social and cultural values. 
Historical injustices from colonialism and slavery, along with continuing challenges of racism, oppression, and economic disadvantage, 
have perpetuated a society in which students of color have been provided with less, yet more is expected of them (Mueller & Broido, 
2012). Inequalities in the educational pipeline begin in early childhood, when underrepresented minority groups are less likely to 
participate in early education programs, and those who do participate tend to be in underresourced programs (Flores & Oseguera, 2013). 
These disparities reduce students’ likelihood of future academic success, given that quality early education programs have been associated 
with positive effects on longer term outcomes, including high school graduation and enrollment in college. Differences in the quality of 
education experiences continue into middle school, where access and participation in accelerated courses, particularly in math, vary 
across demographic groups. In high school, students of color face additional challenges due inequitable education opportunities that 
contribute to racial achievement gaps on high-stakes assessments, as well as differences in curricular intensity and access to advanced 
coursework. Additionally, institutions in both the K-12 and higher education sectors often fail to take responsibility for creating inclusive 
campus environments that foster racially inclusive learning, or for valuing cultural beliefs, traditions, and values (Fish & Syed, 2018; 
Harper & Hurtado, 2007; Muller & Broido, 2012). 


These injustices have led to ethnic/racial achievement gaps in numerous educational outcomes, including college readiness, assignment to 
developmental education, and completion of college-level courses. Bailey et al. (2010) found that Black and Hispanic students tended to 
be assigned to more levels of developmental education courses than their White counterparts. While Hispanic students were as likely as 
White students to complete the developmental education sequence, completion rates remained significantly lower among Black students. 
In another study, Bahr (2010) found that Black and Hispanic students had a greater likelihood of assignment to developmental education, 
which the author primarily attributed to differences in math skills on college entry. Black and Hispanic students were also significantly 
less likely to successfully complete developmental math requirements, further perpetuating racial achievement gaps along the academic 
pipeline. Additionally, studies have found that low-income students receiving Pell Grants who placed into developmental education tend 
to have worse outcomes on persistence, transfer, credits earned, and degree completion relative to developmental education students 
without Pell Grants (Bettinger et al., 2013). These findings may result from a combination of lower quality academic preparation typically 
experienced by low-income students, and financial challenges related to paying for college expenses beyond the Pell Grant that may harm 
postsecondary outcomes for low-income students. 


A limitation to much of the prior research on developmental education is that most rigorous analyses use regression discontinuity designs, 
which only provide estimates of program impacts for students who score just below college-ready on placement tests. These results may 
not generalize to lower performing students who enter college with substantially weaker academic preparation. Boatman and Long (2018) 
addressed this issue by examining the effects of developmental education separately for students assigned to multiple levels of 
developmental education. They found large negative effects on credits earned by Year 3 for students assigned to courses one level below 
college level, ranging from seven credits fewer in reading to 11.5 fewer credits in math, compared with students in the college-level 
course. However, the authors found no significant differences in credits earned in the third year for students in the lowest level of 
developmental education compared with those in the next developmental course. Further, the results revealed even some small positive 
effects on persistence and degree completion for students assigned to the lowest developmental level. The authors concluded that a one- 
size-fits-all approach to developmental education may not consider the unique needs of students with different levels of preparation, given 
that some students may benefit from the additional review of skills needed for college-level courses. 


REFORMS TO DEVELOPMENTAL EDUCATION 


In response to concerns about the high costs, limited effectiveness, and inequitable racial achievement gaps in developmental education, 
many state and higher education systems began implementing reform efforts in the mid to late 2000s (Jaggars & Bickerstaff, 2018). Early 
reform efforts tended to involve isolated practices, such as providing additional advising services, tutoring, learning communities, or 
success courses. These short-term interventions had predominately small effects, and colleges may not have integrated them effectively 
with other student support services. A second wave of reforms began around 2010 and focused more on wide-scale adoption of practices 
that include developing improved polices for course placement (e.g., consideration of multiple measures, providing preparation before the 
exam, and lowering or removing developmental cut scores); providing accelerated instructional modalities for developmental education 
courses (e.g., compressed or corequisite course); and making changes to the pedagogy and content of developmental education courses 
(e.g., contextualization of math or literacy skills with disciplinary content). These more recent reforms have resulted in gains in student 
success on short-term outcomes such as enrollment and passing rates in introductory college-level courses, although they have proved 
difficult to scale up and have had limited impacts on longer term outcomes such as completion rates (Jaggars & Bickerstaff, 2018). 


The first component of Florida’s reform made placement testing and developmental education courses optional for high school graduates 
with a standard diploma who entered a Florida public high school after 2003-04, as well as for active duty military personnel. This idea 
of allowing students to opt out of developmental education is not new; an experiment in the 1960s at Miami-Dade College found that 
students with low test scores who were randomly assigned to college-level courses performed similarly to their peers assigned to 
developmental education (Losak, 1972). Despite this evidence, FCS institutions continued to require developmental education courses 
until the passage of SB 1720 nearly a half-century later. Limited research exists on the effects of removing developmental education 
requirements. However, a natural experiment at a community college that accidently allowed students to enroll in developmental or 
college-level math courses demonstrated some positive effects of self-placement on outcomes such as completion of math degree 
requirements (Kosiewicz & Ngo, 2019). 


The second component of Florida’s reform required that colleges offer remaining developmental education courses using one or more 
innovative instructional strategies. Corequisite courses allow students to complete a developmental course and a gateway course in the 
same subject area simultaneously. Contextualized courses present content in an applied manner relative to students’ meta-majors, or 
groups of similar majors. Modularized courses provide students with a diagnostic exam and then allow them to complete modularized 
instruction for only areas that they have not yet mastered on the exam. Compressed courses meet for a greater number of hours per week 
over fewer weeks, which may allow students to complete two compressed courses in the semester. A growing body of evidence suggests 
that these types of accelerated instructional formats have resulted in improvements in student outcomes, including enrollment and 
completion rates of gateway courses, persistence, and degree attainment (Denley, 2015; Hodara & Jaggars, 2014; Jaggars et al., 2014; 
Kalamkarian et al., 2015; Okimoto & Heck, 2015). 


Colleges in Florida had flexibility in selecting which instructional strategies to offer for developmental education courses. A review of 
institutional plans approved by the Division of Florida Colleges at the beginning of the reform revealed that modularized and compressed 
options were the most commonly adopted strategies, with all 28 FCS institutions offering these strategies in at least one subject area (Hu 
et al., 2014). The plans identified contextualized courses as the least popular, with only seven institutions offering any courses using this 
modality. Yet even among colleges offering the same modality, the plans revealed differences in implementation in terms of the type of 
technology used in classes, the use of diagnostic exams, the number of credit hours per course, the number of weeks of instruction, and 
plans for redesigning instruction. 


The third component of Florida’s reform required colleges to develop plans to provide enhanced advising and academic support services. 
A survey of lead administrators at FCS institutions revealed that colleges made extensive changes, which included implementing early 
alert systems to follow up with at-risk students, increasing the amount of time that advisors spend with students, and adding more online 
orientation resources (Hu et al., 2017). College students who receive enhanced advising tend to take and pass more courses (Scrivener & 
Weiss, 2009) and have an increased likelihood of taking advantage of tutoring and other types of academic support services (Visher et al., 
2010). Additionally, academic services such as tutoring and supplemental instruction may be particularly effective at better serving the 
needs of Black students (Barhoum, 2018) and Indigenous students (Guillory, 2009). 


PRIOR RESEARCH ON THE EFFECTIVENESS OF FLORIDA’S SB 1720 


A growing body of research indicates that Florida’s SB 1720 has resulted in significant improvements in first-year coursetaking outcomes 
in math and English. Participation in developmental education declined sharply in the first year of the reform, particularly in math, for 
which developmental coursetaking rates in the first year decreased from 38% in fall 2013 to 22% in fall 2014 (Hu et al., 2016). Students 
had an 8-percentage-point increase in the likelihood of enrolling in a college-level math course, and a 4-percentage-point increase in the 
likelihood of completing college-level math in the first year. The research has also revealed differences in the effectiveness of the reform 
by student characteristics. Black and Hispanic students experienced greater improvements in first-year coursetaking outcomes relative to 
White students (Park et al., 2018). Reform impacts also differed by students’ high school academic preparation, with greater 
improvements in first-year coursetaking outcomes among students from the two lowest academic tracks (Park-Gaghan et al., 2019). 
Additionally, we examined how changes in college coursetaking before and after the reform altered costs of completing introductory 
college-level courses in math and English for students and institutions, and whether cost savings differed by racial/ethnic subgroup. The 
results indicated that Florida’s developmental education reform has contributed to short-term cost savings for both students and 
institutions and has helped to reduce racial/ethnic gaps in the costs to the short-term outcome of gateway course completion (Mokher, 
Park-Gaghan, & Hu, 2019). 


The present study makes two additional contributions to further advancing our understanding of the impacts of Florida’s reform. First, 
given the early success of the reform in improving gateway coursetaking outcomes, we now seek to determine whether positive effects of 
the reform may also be detected among other more distal student outcomes. Belfield et al. (2019) described a growing emphasis on the 
importance of early momentum metrics (EMMs) as predictors of longer term student success, thus serving as leading indicators of 
effectiveness for college reform efforts. Their study using data from community colleges in three states found that nine indicators of credit 
momentum, gateway course momentum, and persistence momentum strongly predict longer term completion rates. In addition, they 
found that “EMMs strongly predict longer-term completion rates not only for students generally but also for Black and Hispanic students, 
suggesting that college efforts to close racial equity gaps in early momentum represent a crucial step toward closing gaps in credential 
completion” (p. 10). Because credit accumulation is a strong predictor of progress to degree completion, we examine the number of 
college-level credits attempted and earned in both the short term (after one year) and long term (by the end of Year 3). Examining these 
outcomes is important given the large number of students who leave community colleges before acquiring enough credits to complete 
postsecondary credentials, even though they would likely benefit from higher levels of employment and greater wages if they completed a 
community college degree, diploma, or certificate (e.g., Jepsen et al., 2014). 


The second contribution of this study is to further explore the reform’s impact on the success of students from a broad range of 
background characteristics, including race/ethnicity, income, and prior academic performance. Notably, we expand the investigation of 
differential effects beyond Black and Hispanic students to also explore differences among Asian/Pacific Islander, Indigenous, multiracial, 
and other racial/ethnic groups. Indigenous populations in particular remain understudied in the field of higher education, which has 
resulted in a lack of knowledge and understanding about this population (Shotton et al., 2013). Even though higher education enrollments 
among American Indians and Alaska Natives have more than doubled since the 1980s, Indigenous students remain underrepresented in 
higher education, and colleges have not done well in recruiting and retaining them (Brayboy et al., 2012). Our analyses will shed new 
light on whether existing racial achievement gaps improve or decline following Florida’s reform efforts. 


STUDY HYPOTHESES 


We present four hypotheses for how the reform may accelerate students’ progression through postsecondary education. Our first 
hypothesis states that students will enroll in more for- credit college courses during the first year because they are either exempt from 
developmental education courses or have the option to enroll in developmental education through instructional modalities that students 
may complete more quickly than traditional developmental education courses. Students who opt out of developmental education courses 
and enroll directly into college-level courses will attempt more credits in the first semester by default because developmental education 
courses do not count for college-level credit. Yet students who opt in to developmental education and nonexempt students may also end 
up taking more credits because the reform required colleges to offer developmental courses using new instructional modalities intended to 
more closely align with students’ needs and to accelerate students’ progression to college-level courses. 


Our second hypothesis states that credit accumulation in the first year will also increase, because colleges started making changes to 
instructional practices to both developmental and gateway courses, and providing additional institutional support services to help students 
succeed in their courses. In a 2017 survey of FCS institutional leaders, more than half of respondents indicated that their colleges made 
changes to pedagogy for developmental and gateway courses following SB 1720 (Hu et al., 2017). The reform also required colleges to 
provide students with enhanced advising and academic support services. While each college had discretion on the specific types of 
support services to provide, some common approaches included expanding hours for success centers or tutoring, adding or increasing peer 
supplemental instruction leaders, mandating student life skills courses to assist students with a better understanding of college, providing 
boot camps or training sessions to engage students in intensive skill building, and implementing early alert systems to identify and 
support academically at-risk students. Colleges provided most of these additional support services to all students, regardless of whether 
they enrolled in developmental education courses. Previous research indicates that these types of supports may help to improve student 
success on outcomes such as persistence and degree completion (Bers & Younger, 2014; Gallard et al., 2010; Tampke, 2013). 


Our third hypothesis states that the initial success in for-credit courses will set students on a more successful long-term trajectory, 
resulting in a greater number of credits attempted and earned in subsequent years. We may expect these longer term improvements 
because of students having fewer opportunities to drop out, and improved alignment of developmental education with college-level 
courses. Additionally, the accomplishments from earlier credit accumulation can create “academic momentum” that leads students to 
develop self-efficacy and academic self-concept, resulting in greater commitment to degree completion (Attewell & Monaghan, 2016; 
Attewell et al., 2012). This may be particularly relevant for exempt students, because assignment to developmental education has 
traditionally been associated with stigma and discouragement (Colyar & Stich, 2011; Deil-Amen & Rosenbaum, 2002). 


Our final hypothesis states that the increase in the number of credits attempted and earned will be greatest among those most likely to be 
assigned to developmental education before the reform. This includes students from underrepresented racial/ethnic groups, low-income 
students, and students with low levels of academic preparation in high school—all of whom stand to benefit more if the developmental 
education reform is effective. Additionally, prior research has shown that additional advising and support services, similar to those 
provided under Florida’s reform, tend to be particularly effective for students of color students (Barhoum, 2018; Guillory, 2009). Given 
that Florida’s reform has already resulted in greater gains on first-year coursetaking outcomes for Black and Hispanic students (relative to 
White students) and students with low levels of high school academic preparation (Park et al., 2018; Park-Gaghan et al., 2019), these 
differential gains may continue to progress into subsequent postsecondary outcomes. 


RESEARCH DESIGN 


Developmental education in Florida is offered almost exclusively by state colleges in the FCS, so the reform has focused on these 
institutions. The FCS consists of 28 public community colleges with an estimated enrollment of approximately 80,000 students among 70 
campuses across the state. FCS institutions offer a variety of programs, including bachelor’s degrees, associate’s degrees in arts and 
science, college credit certificates, vocational credits, college and vocational preparatory programs, and lifelong learning (Florida College 
System, 2017). Florida has a separate state university system for four-year universities offering undergraduate and graduate degrees; 
nearly all students in these universities assigned to take developmental courses receive a referral to one of the FCS institutions for the 
courses. 


Before Florida’s developmental education reform, all FCS institutions had to administer a placement test to incoming students and assign 
them to up to two levels of developmental education courses if they scored below college-ready in reading, writing, or math. All FCS 
institutions began implementing the reform during the 2014—15 academic year, and implementation has continued in each of the 
subsequent years to date. As a result, there is no comparison group of colleges within the state that did not participate in the reform’s 
changes to developmental education. Instead, we rely on an ITS design to provide a quasi-experimental estimate of the reform’s impact on 
student outcomes over time. 


DATA 


We used student-level records provided by Florida’s K-20 Education Data Warehouse. These data allowed us to track the population of 
students enrolled in public colleges statewide and were linked to students’ K-12 records if they attended a Florida public high school. 
The types of data include student demographic characteristics, high school transcript records, college enrollment records, and college 
transcript records. 


Florida’s developmental education reform would likely have impacts on a wide range of students. However, for the purpose of having 
clear and comparable student populations before and after the reform, we used FTIC students to evaluate the impacts of the policy on 
student progress and success. Our sample includes all FTIC students enrolled in all 28 public state colleges. We included three cohorts 
of students before the reform (those entering in the fall of 2011, 2012, and 2013) and up to three cohorts of students after the reform 
(those entering in the fall of 2014, 2015, and 2016). Each cohort consists of approximately 70,000 students. The total sample sizes 
resulted in N = 407,309 for the Year 1 outcomes with six cohorts and N = 273,298 for the Year 3 outcomes with four cohorts. 


Our outcome variables, measured one and three years following initial college enrollment, represent continuous indicators for the 
number of college-level credits attempted and the number of college-level credits earned. The first-year outcomes include two semesters 
from the fall to the spring of the first year of enrollment, and the three-year outcomes include eight semesters from the fall of Year 1 to 
spring of Year 3, including summer semesters between academic years. Developmental education courses do not count for college credit 
and therefore do not count toward the number of college-level credits either attempted or earned. If a student did not enroll in any 
courses in any of the subsequent semesters after initial entry, they remained in the sample with zero credits attempted and earned during 
the semester(s) of nonenrollment. 


Our independent variables of interest included student background characteristics for race/ethnicity (Asian/Pacific Islander, Black, 
Hispanic, Indigenous, multiracial, White, and other/unknown), sex, and FRL status. We had no missing data for any of these variables. 
We also included independent variables for high school academic preparation, using two categorical variables based on students’ high 
school transcript records from math and English courses. In math, we classified students as on a “basic” track if they did not 
successfully complete algebra II, a “standard” track if they completed algebra II but no advanced math classes beyond that, and an 


“advanced” track if they completed algebra II plus at least one more advanced course. ! During the time frame of our analysis, students 
had to complete four credits in math for high school graduation, including algebra I and geometry. Students did not have to take algebra 
II to graduate, but we considered whether students had completed this course because the literature has identified it as an important 
gatekeeper course for success in early college outcomes (Gaertner et al., 2014; Kim et al., 2015; Long et al., 2009). In English, we 


classified students as on a “basic” track if they ever enrolled in a remedial English course,” a “standard” track if they completed only 
regular or honors-level English courses, and an “advanced” track if they ever completed an English course that could result in college 
credit (Advanced Placement, International Baccalaureate, or dual enrollment). About 22% of students had missing values for these 
variables because they did not have high school transcript records in the administrative data. In these cases, we used the dummy variable 
adjustment method where we set the value of the missing independent variable to a constant value of zero and included an additional 
dummy variable to the model to indicate whether the actual value was missing (Cohen & Cohen, 1983). In the models that examined 
whether the reform had differential effects by prior academic preparation, we limited this subgroup analysis to students who had high 
school transcript records. All models also included a dichotomous post-reform indicator (1 = post-reform, 0 = pre-reform), a continuous 
cohort variable to account for any underlying temporal trends, an annual indicator for local employment rate in the county of the college 
attended, and institutional fixed effects for the college of initial enrollment. 


Table 1 presents summary statistics by cohort on first-year college coursetaking, student background characteristics, high school 
academic indicators, and sample size. After the reform, developmental education enrollments declined in first-year coursetaking, 
particularly in developmental math, which decreased from 43.71% in 2013 (the last cohort pre-reform) to 26.35% in 2014 (the first post- 
reform cohort). Enrollments in college-level courses increased by 6.5 percentage points in English and 8.4 percentage points in math 
during the first year of the reform and continued to increase slightly thereafter. These changes in first-year coursetaking indicated that 
we should expect increases in the number of first-year credits attempted after the reform. 


It is also important to note that there were some shifts in student background characteristics over time. The percent of Hispanic students 
increased from 30.56% in 2011 to 36.44% in 2016, with corresponding declines among White and Black students. Fewer than 3% of 
students identified as Asian/Pacific Islander, Indigenous, multiracial, or other/unknown. The population of FRL students increased from 
30.88% in 2011 to 41.57% in 2016. Across all cohorts, the majority of students only completed a basic track in high school English or 
math. Over time, the percent of students on the advanced track during high school grew, particularly in English, which increased from 
12.97% in 2011 to 17.15% in 2016. We included all these student background and high school academic preparation variables in the ITS 
models to account for changes in the student population over time. 


Table 1. Summary Statistics on First Year Coursetaking, Student Background Characteristics, High School Academic Indicators, 
and Sample Size 


Cohort of college entry year 


Variable 2011 2012 2013 2014 2015 2016 
First-year college coursetaking 
% Took developmental reading 25.39 22.65 22.77 11.44 8.49 6.58 


% Took developmental writing 22.81 18.86 18.54 13.28 10.98 10.18 


% Took developmental math 47.56 47.86 43.71 26.35 23.18 21.44 


% Took for-credit English 56.15 60.82 63.67 70.18 73.06 72.43 
% Took for-credit math 40.10 42.76 48.28 56.66 62.29 61.31 
Student background characteristics 
% Asian/Pacific Islander 2.51 2.59 2.51 2.54 1.66 2.61 
% Black 22.52 20.89 21.82 20.74 20.04 19.36 
% Hispanic 30.56 31.30 32.44 34.38 34.95 36.44 
% Indigenous 0.33 0.34 0.31 0.31 0.16 0.30 
% Multiracial 2.06 2.40 2.54 2.86 1.84 3.26 
% White 41.09 41.80 39.61 38.49 38.16 34.92 
% Other/unknown 0.93 0.67 0.71 0.68 3.20 3.10 
% Female 52.58 52.30 51.98 52.66 52.24 52.19 
% Free and reduced-price lunch 30.88 31.81 3712 39.73 41.62 41.57 
High school academic indicators 
% Basic English 50.76 49.85 50.84 52.85 49.92 50.24 
% Standard English 36.27 35.50 34.92 32.30 32.91 32.61 
% Advanced English 12.97 14.65 14.24 14.85 17.17 17.15 
% Basic math 56.62 54.83 57.02 56.92 54.07 54.14 
% Standard math 14.79 14.15 15.83 13.70 14.54 13.38 
% Advanced math 28.68 31.01 27.15 29.38 31.39 32.47 
Sample size 
N students 72,468 64,757 68,335 68,246 67,114 66,925 
N colleges 28 28 28 28 28 28 
METHODS 


We assess the impact of the reform by comparing the number of credits attempted (or earned) after the reform relative to any prepolicy 
trends. The main assumption of ITS is that the outcomes would continue under the prepolicy trend in the absence of the policy. This 
assumption may be violated if any cohort-specific changes over time exist, such as changes to the composition of students attending FCS 
institutions, the introduction of other educational policies affecting the same cohorts, or changes to the labor market. We took several 
measures to reduce this limitation of the ITS. First, our detailed administrative records allowed us to account for a large set of student 
characteristics, including race/ethnicity, sex, and FRL status in K—12 as a proxy for family income status. We also used student high 
school transcript data to create indicators for academic preparation in high school to account for any changes over time in coursetaking 
before college enrollment. The ITS models also included institutional fixed effects for the first college of enrollment to control for any 
changes over time in the distribution of students to various colleges. To further account for any potential changes over time independent 
of the reform, we controlled for the local unemployment rate. Following the Great Recession, enrollments in community colleges 
increased by nearly 25% between 2008 and 2011 as workers sought to gain new skills to improve their employment prospects (G. Chen, 
2019). These enrollments began to decline after 2012 because of more stabilized job outlooks. We included annual measures of county- 
level unemployment rates to control for these differences in economic conditions over time and across different regions of the state. 


Our student-level data set includes three years of data pre-reform and three years of data post-reform for the outcomes of credits 
attempted and earned in the first year of college. Not enough time has elapsed to observed three-year outcomes for all the postpolicy 
cohorts, so we had to limit these longer term analyses to three years of prepolicy data relative to a single year of postpolicy data. Although 
this analysis is not as rigorous, it provides some initial evidence of whether the reform may have longer term impacts. 


We estimated the overall impact of the reform on credits attempted and earned for student i at college j in year (cohort) ¢ using the 
following model: 


vit = Bo + By(2014); + Bo(S)ije + B3(AS) ijn + G + At + eit (1) 


In this specification, 6; captures the change in the number of credits attempted and earned in the post-reform period, /, represents a 
vector of student background characteristics, 83 represents a vector of high school academic preparation indicators, ¢; represents a college 
fixed effect to account for unobserved heterogeneity across institutions, and 4, represents a continuous year (cohort) indicator to account 
for any underlying temporal trends. The error term ¢;;, is clustered by college. 


To explore whether there were differential changes in outcomes by student subgroups, we ran an additional set of models that included 
interaction terms between the subgroup variables and the postpolicy variable. In the model that follows for racial/ethnic subgroups, we 
included indicators for Asian/Pacific Islander, Black, Hispanic, Indigenous, multiracial, and other/unknown students (White students form 
the comparison group) and interacted these indicators with the post-reform indicator. We estimated the following model for student i at 
college j in year (cohort) ¢: 


Vit = Bo + Bi(2014) + Bx(Asian) + B3(Black);, + By(Hispanic);, + Bs(Indigenous) + Bg(Multi) , + B7(Other) y+ B3(2014*Asian) i + 
Bo(2014*Black) i, + B 19(2014 *Hispanic) jj, + B 1)(2014*Indigenous) ji, + B12(2014*Multi) , + B13(2014*Other) i + Bi 4(S)ije + Bi s(AS) iit 


+ HAs + bi 


(2) 


Under this specification, 8; captures the change in credits attempted/accumulated in the post-reform period, > through 7 capture the 
overall difference in outcomes for each racial/ethnic group, /s through £3 capture any differential changes in student outcomes by 
race/ethnicity post-reform, £74 represents a vector of student background characteristics, £75 is a vector of high school academic 
preparation indicators, ¢; represents a college fixed effect to account for unobserved heterogeneity across institutions, and /, represents a 


continuous year (cohort) indicator to account for any underlying temporal trends. This model captures the base changes in overall student 
outcomes between the pre-reform period and the post-reform period, as well as whether differential effects exist for students in each 
racial/ethnic group. We also ran separate models in which we replaced the race interactions with interaction terms between the post- 
reform variable and (1) the FRL status indicator (where non-FRL students served as the reference group), or (2) a series of dichotomous 
variables for the level of high school academic preparation in math and English (where a “standard” high school track served as the 
reference group). These models examined whether the reform had differential impacts by family income status or level of academic 
preparation. 


RESULTS 


We begin by presenting descriptive analyses of changes in first-year credit accumulation patterns over time for the three prepolicy cohorts 
(2011-2013) and the three postpolicy cohorts (2014-2016). As shown in Figure 1, the number of credits attempted in the first year 
increased during each of the three prepolicy cohorts, with a larger increase after the introduction of the reform in 2014, from 14.5 to 15.7 
credits (a difference of 1.2 credits). Outcomes continued to rise slightly in each of the subsequent postpolicy years. We also found that the 
ratio of credits earned to unearned remained relatively consistent over time, with students successfully earning credit for about 70% of the 
credits that they attempted. 
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Figure 1. Average number of college-level credits attempted in the first year, including credits earned and credits not earned 


Figure 2 shows cumulative credits attempted and earned in each semester over three years for the pooled prepolicy cohorts (2011-2013) 
relative to the first postpolicy cohort (2014). The postpolicy cohort had slightly more credits attempted and earned in the first semester 
(about 0.6 credits), and this difference continued to slowly increase in subsequent semesters. By the end of the third year, students in the 
postpolicy cohort had attempted an average of 2.4 additional credits and earned 1.8 additional credits. 
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Figure 2. Cumulative college-level credits attempted and earned in each semester by the end of Year 3 
OVERALL REFORM IMPACTS ON CREDITS ATTEMPTED AND EARNED 


The regression-adjusted results from the ITS analyses examining outcomes of credits attempted and earned for all students appear 
similar to the results from the descriptive analyses. The first set of results in Table 2 shows the overall impact of the reform on credits 
attempted and credits earned in Years | and 3 for all students. In three of the four models, we found positive and statistically significant 
impacts for the underlying time trend, ranging from 0.452 credits earned in Year | to 1.387 credits attempted in Year 3. This indicates 
that all the credit accumulation outcomes were improving before the reform began. Yet we also found that, even after controlling for 
these preexisting time trends and changes in student populations, outcomes continued to improve at an even greater rate after the reform. 
Among first-year coursetaking outcomes, the reform resulted in an increase of 0.910 credits attempted and 0.312 credits earned. We also 
continued to see positive impacts into the third year of college enrollment, with an increase of 0.828 credits attempted and 0.464 credits 
earned after the reform. Although all these post-reform increases are statistically significant, they are relatively small in magnitude, at 
less than one credit. 


Table 2. Interrupted Time Series Results of the Reform Effects on College-Level Credits Attempted and Earned in Years 1 and 3, 
All Students 


Credits Credits Credits Credits 
attempted, earned, attempted, earned, 
Year 1 Year 1 Year 3 Year 3 
Time 0.643 * 0.452 1.387 + 1.248 BEE 
(0.254) (0.353) (0.353) (0.36) 


Post-reform 0.910 0.312 0.828 0.464 


(0.108) (0.123) (0.182) (0.153) 
R2 0.169 0.138 0.116 0.111 
N cohorts 6 6 4 4 

N colleges 28 28 28 28 

N students 407,309 407,309 273,298 273,298 


Note. All models include variables for student background characteristics, high school academic indicators, local 
unemployment rate, and institutional fixed effects. Standard errors, which are clustered by college, are shown in 
parentheses. 


*p < 05. **p < 01. ***p < 001. 
DIFFERENTIAL REFORM IMPACTS BY RACE/ETHNICITY 


The second set of results examined whether the reform had differential impacts by students’ race/ethnicity. Figure 3 displays average 
values for the credit accumulation outcomes for each subgroup pre- and post-SB 1720. Across all four outcomes, Asian/Pacific Islander, 
multiracial, and White students attempted and earned a greater number of credits compared with Black, Indigenous, and other students 
in the prepolicy period. Hispanic students had similar outcomes (within 1 credit hour) to White students on all outcomes in the prepolicy 
period. Yet Figure 3 also reveals variation by race/ethnicity in terms of the changes in student outcomes between the pre- and post-SB 
1720 cohorts. While outcomes for multiracial students remained similar or even slightly declined between the pre- and postperiods, 
other subgroups, such as Black students, consistently demonstrated greater gains in student outcomes over time. 
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Figure 3. Average number of college-level credits attempted and earned by the end of Years 1 and 3, by race/ethnicity 


We further explored these results in our regression analyses that included interaction terms between race/ethnicity and the postpolicy 
indicator (Table 3). Because of the small sample sizes for some of the racial/ethnic subgroups, we considered statistical significance at 
the 10% level. In the post-reform period, we observed a base increase of 0.370 credits attempted and 0.107 credits earned in the first 
year for all students, regardless of race/ethnicity. Although first year credit accumulation was, on average, lower for Black, Hispanic, 
and Indigenous students as compared with White students, we observed greater rates of change for these subgroups in the postpolicy 
period (1.588 credits for Black students, 0.616 credits for Hispanic students, and 0.809 credits for Indigenous students). Put differently, 
all students showed increases in the number of credits accumulated over the first year in the postpolicy period, with Black, Hispanic, and 
Indigenous students having greater increases than White students. We also found that the reform had smaller effects on short-term 
outcomes for multiracial students relative to White students. Multiracial students achieved slightly higher credits attempted and earned 
in the first year in the prepolicy period, and both groups performed similarly (within about a half-credit) in the post-reform period. 


The differential impacts of the reform for Black and Hispanic students seemed to diminish in the third year of coursetaking. The reform 
was associated with an additional 0.828 credits attempted for Black students and 0.492 credits attempted for Hispanic students in Year 3. 
However, we found no statistically significant differences in the impact of the reform by race in the number of credits earned in the third 
year. Yet for Indigenous students, the results suggested some evidence of performance gains over time. The reform was associated with 
an additional 3.552 credits attempted and 3.102 credit earned in the third year for Indigenous students relative to White students. We 
also found that multiracial students tended to attempt and earn slightly more credits relative to White students in the base estimates, but 
the impact of the reform was slightly less for multiracial students. 


Table 3. Interrupted Time Series Results of the Reform Effects on College-Level Credits Attempted and Earned in Years 1 and 3, 
by Racial/Ethnic Subgroup 


Credits Credits Credits Credits 
attempted, eared, attempted, earned, 
Year 1 Year | Year 3 Year 3 
Time 0.639 one. 0.447 are 1.403 *3* 1.260 a 
(0.039) 0.040 (0.289) (0.270) 
Post-reform 0.370 wee 0.107 ~ 0.485 bs 0.373 ~ 
(0.061) (0.062) (0.246) (0.230) 
Asian/Pacific Islander 0.493 eet 1.424 ie 8.166 ate 8.171 nae 
(0.113) (0.116) (0.361) (0.338) 
Black -2.852 + -2.654 ee -4,.450 ane -5.085 *%* 
(0.048) (0.049) (0.155) (0.145) 
Hispanic -0.660 +h -0.164 ane 1.856 aie 1.521 ete 
(0.042) (0.043) (0.136) (0.128) 
Indigenous -1.247 babs -1.465 eae -4,236 une -4,285 ane 
(0.308) (0.316) (0.981) (0.917) 
Multiracial 0.565 +1 -0.102 0.693 ~ -0.671 ~ 
(0.118) (0.121) (0.376) (0.352) 
Other/unknown -0.005 0.148 0.483 1.074 ~ 
(0.202) (0.207) (0.647) (0.605) 
Post*Asian/PI 0.247 0.134 0.645 0.767 
(0.165) (0.170) (0.725) (0.678) 
Post*Black 1.588 ie 0.540 ane 0.828 * 0.067 
(0.068) (0.070) (0.303) (0.283) 
Post*Hispanic 0.616 ba 0.293 she 0.492 * 0.212 
(0.060) (0.061) (0.263) (0.246) 
Post*Indigenous 0.809 ~ 0.581 3.552 ~ 3.102 ~ 
(0.467) (0.479) (1.996) (1.866) 
Post*Multiracial -0.371 7 -0.685 eae -1.621 * -1.526 * 
(0.163) (0.167) (0.703) (0.657) 
Post*Other/unknown 0.281 0.313 1.259 0.743 
(0.235) (0.241) (1.356) (1.267) 
R 0.1704 0.1395 0.1173 0.113 
N Cohorts 6 6 4 4 
N Colleges 28 28 28 28 
N Students 411,688 411,688 276,697 276,697 


Note. All models include control variables for student background characteristics, high school academic 
indicators, local unemployment rate, and institutional fixed effects. Standard errors in parentheses. 
~p <.10. *p < .05. **p < .01.***p < .001. 


DIFFERENTIAL REFORM IMPACTS BY FREE OR REDUCED-PRICE LUNCH STATUS 


The third set of results examined differential impacts of the reform by FRL status (Table 4). Among all students post-reform, there was a 
base increase of 0.481 credits attempted and 0.178 credits earned in the first year. FRL students improved their outcomes at faster rates 
than non-FRL students, narrowing preexisting achievement gaps. After the reform, FRL students attempted an additional 1.115 credits 
and earned an additional 0.348 credits in the first year. 


Yet, as with some of the racial/ethnic subgroup analyses, we found that the differential effects of the reform by FRL status diminished by 
the end of Year 3. Post-reform, there was a base increase of 0.764 credits attempted and 0.611 credits earned in three-year coursetaking 
outcomes. We found no significant differences in program impact by FRL status in the number of credits attempted or earned in Year 3. 


Table 4. Interrupted Time Series Results of the Reform Effects on College-Level Credits Attempted and Earned in Years 1 and 3, 
by Free or Reduced-Price Lunch Status 


Credits attempted, Year Credits earned, Credits attempted, Credits earned, 


1 Year | Year 3 Year 3 
Time 0.626 * 0.447 1.388 are 1.245 +e 
(0.242) (0.320) (0.354) (0.307) 
Post-reform 0.481 ia 0.178 : 0.764 ay 0.611 a 
(0.156) (0.017) (0.222) (0.202) 
FRL -0.662 bie -0.643 oat -2.963 ane -2.614 4% 
(0.067) (0.059) (0.086) (0.0884) 
Post*FRL 1.115 ee, 0.348 bi 0.162 -0.369 
(0.204) (0.158) (0.161) (0.186) 
R2 0.170 0.138 0.116 0.111 
N Cohorts 6 6 4 4 
N Colleges 28 28 28 28 
N Students 407,309 407,309 273,298 273,298 


Note. All models include variables for student background characteristics, high school academic indicators, local 
unemployment rate, and institutional fixed effects. Standard errors, which are clustered by college, are shown in 


parentheses. 


*p < 05. **p < 01. ***p < 001. 


DIFFERENTIAL REFORM IMPACTS BY ACADEMIC PREPARATION 


The last set of estimates examined heterogeneity in the reform’s impacts based on students’ level of academic preparation in high school 
math and English. These models included interaction terms between the post-reform indicator and dichotomous variables for whether 
the student completed a basic or advanced coursetaking track in high school math or English, relative to the omitted group of students 
who completed a standard track in the corresponding subject area. We found preexisting achievement gaps, with students on the basic 
English track attempting 2.433 fewer credits in Year 1 and 3.641 fewer credits in Year 3 compared with students on the standard high 


school track (Table 5). Yet the reform had greater positive effects for students on the basic English track, resulting in an additional 1.556 
credits attempted in Year 1, 0.411 credits earned in Year 1, and 1.180 credits attempted in Year 3. We also found large base effects of the 
advanced English track relative to the standard English track, with approximately 2.5 more credits attempted and earned in the first year 
and six more credits attempted and earned in the third year. Yet the reform had smaller impacts on the number of credits attempted and 
earned in the first year for students on the advanced track relative to the standard track. 


Table 5. Interrupted Time Series Results of the Reform Effects of College-Level Credits Attempted and Earned in Years 1 and 3, 
by High School English Track 


Credits Credits Credits Credits 
attempted, earned, attempted, earned, 
Year | Year 1 Year 3 Year 3 
Time 0.624 ' 0.406 1.568 + 1.361 “e 
(0.240) (0.318) (0.525) (0.445) 
Post-reform 0.389 ae 0.219 -0.012 0.101 
(0.119) (0.132) (0.214) (0.194) 
Basic English track -2.433 wee 1,907 *** -3.852 wee 3.641 
(0.047) (0.044) (0.132) (0.110) 
Advanced English track 2.457 wee 2.503 one 6.383 one 6.137 #ee 
(0.51) (0.075) (0.184) (0.209) 
Post* Basic 1.556 wee 0.411 +e 1.180 ie 0.237 
(0.113) (0.110) (0.316) (0.333) 
Post* Advanced -0.826 *ee 0.650 *** -0.108 -0.096 
(0.0991) (0.106) (0.432) (0.454) 
R2 0.176 0.167 0.138 0.140 
N Cohorts 6 6 4 4 


N Colleges 28 28 28 28 

N Students 315,799 315,799 210,810 210,810 
Note. All models include variables for student background characteristics, high school academic indicators, local 
unemployment rate, and institutional fixed effects. Standard errors, which are clustered by college, are shown in 
parentheses. The “standard” English track is the reference group. Students missing high school transcript records are 
excluded. 

*p < 05. **p < 01. ***p < 001. 


A similar pattern emerged in math, with large baseline differences in credits attempted and earned by high school academic track (Table 
6). Relative to students on the standard math track, students on the basic track attempted 3.600 fewer credits and earned 2.651 fewer 
credits in the first year. Yet as with English, the reform had a larger impact for students on the basic track, with differential effects of 
1.278 credits attempted and 0.662 credits earned in the first year. Among the Year 3 outcomes, students on the basic track attempted 
8.590 fewer credits and earned 6.480 fewer credits relative to those on the standard track. The reform continued to have larger effects for 
students on the basic track, with an additional 1.808 credits attempted and 1.107 credits earned in Year 3. We also found that, as 
expected, students on the advanced math track attempted and earned more credits in Years 1 and 3 relative to students on the standard 
track. The reform had slightly smaller positive effects for these advanced students in Year 1, which matched our expectations because 
students in this group were less likely to be affected by the policy change, given that they had a lower likelihood of assignment to 
developmental education before the reform. However, in Year 3, we found no statistically significant differences in the reform’s impact 
among students in the standard and advanced tracks. 


Table 6. Interrupted Time Series Results of the Reform Effects on College-Level Credits Attempted and Earned in Years 1 and 3, 
by High School Math Track 


Credits Credits Credits Credits 
attempted, earned, attempted, earned, 

Year | Year 1 Year 3 Year 3 

Time 0.630 ‘ 0.410 1.581 “t 1.364 ake 
(0.258) (0.328) (0.527) (0.446) 

Post-reform 0.560 *** 0.038 -0.541 -0.590 
(0.129) (0.147) (0.355) (0.324) 

Basic math track -3.600 wee 2.641 -8.590 wee 6.480 Fe 
(0.053) (0.049) (0.153) (0.146) 

Advanced math track 2.511 wee 3.146 7.425 ial 8.225 ba 
(0.047) (0.049) (0.172) (0.174) 

Post* Basic 1.278 *e* 0.662 *** 1.808 ane 1.107 baka 
(0.151) (0.164) (0.292) (0.263) 

Post* Advanced -0.755 we 0.275 - 0.336 0.591 
(0.114) (0.114) (0.291) (0.264) 

R2 0.176 0.168 0.138 0.140 

N Cohorts 6 6 4 4 

N Colleges 28 28 28 28 

N Students 315,799 315,799 210,810 210,810 


Note. All models include variables for student background characteristics, high school academic indicators, local 
unemployment rate, and institutional fixed effects. Standard errors, which are clustered by college, are shown in 
parentheses. The “standard” math track is the reference group. Students missing high school transcript records are 
excluded. 

*p < 05. **p < 01. ***p < 001. 


DISCUSSION 


While many states have begun experimenting with modifications to their developmental education programs, Florida has had a much 
more extensive reform, consisting of a comprehensive set of changes to placement, instruction, and support services. The majority of 
students could opt out of developmental courses and enroll directly in college-level courses, while those remaining in developmental 
education took modularized, compressed, corequisite, or contextualized courses. One of the intents of the reform was to accelerate 
students’ academic progress by removing additional developmental education course requirements or offering modalities that students 
could complete more quickly than traditional semester-long developmental courses. Colleges also provided comprehensive student 
support services to help students succeed while progressing at a more accelerated pace. 


Over the time frame of our analysis, we identified some changes to the characteristics of students and a slight increasing trend in credit 
accumulation outcomes even before the reform. This is likely due to changing economic conditions and student enrollment patterns 
following the recovery from the Great Recession. Yet even after controlling for student characteristics, economic conditions, and 
preexisting time trends, we found statistically significant positive effects on both credits attempted and credits earned in the first and third 
years of college. This indicates that students experienced increased academic momentum in the short term and longer term. Although the 
overall effects were relatively small in magnitude, the findings from this study remain considerable for three reasons. First, the state 


legislature provided no additional appropriations for implementation of the reform. Early in the life cycle of any new required program, 
one often finds low levels of implementation and many obstacles to improvement (e.g., Spillane, 2004). Yet Florida colleges were still 
able to demonstrate early gains in student outcomes, despite the lack of additional resources to support implementation. Our prior work 
has shown that the reform resulted in substantial savings to both students and institutions in the cost of pathways taken to completing the 
first college-level course (Mokher et al., 2019), so it is noteworthy that these gains in efficiency did not come at the expense of efficacy. 


Second, other statewide reforms of developmental education, such as Tennessee’s corequisite policy, have demonstrated large gains on 
first-year coursetaking outcomes but have failed to improve longer term outcomes such as persistence, transfer, or degree completion 
(Ran & Lin, 2019). The authors posited that this lack of influence on longer term outcomes may be attributed in part to the absence of 
additional supports beyond the first course. Florida’s reform is unique in that it did provide considerable enhancements to advising and 
academic support services to better help students succeed. 


Third, Florida’s reform had effects of more than three additional credits (roughly equivalent to one full college-level course) attempted 
and earned for certain subgroups of students, as we will discuss further. This suggests that there are some students for whom the reform 
has been particularly effective. Our findings contribute to a growing body of research intended to improve our understanding of for whom 
and under what conditions educational interventions are most effective (e.g., Reardon & Stuart, 2017). 


Examining differential impacts among student subgroups is particularly important given that critics expressed an initial concern that the 
reform could have unintended consequences that may harm disadvantaged student populations. Yet we found that the reform had even 
greater positive effects for Black, Hispanic, and low-income students, particularly in the first year. Given that prior research has found 
that students in these subgroups are more likely to be assigned to developmental education and more likely to be harmed by this 
assignment (Bahr, 2010; Bailey et al., 2010; Bettinger et al., 2013), they also stand to benefit more if the developmental education reform 
is effective. Yet our results also indicate that these subgroup effects diminished by the third year, so it will be important to continue to 
monitor these long-term outcomes to see if these trends persist beyond the first postpolicy cohort. The number of credits attempted and 
earned in the first year did rise for the second and third postpolicy cohorts, so it’s possible that some initial challenges to implementation 
may improve in subsequent years. However, if these subgroup trends continue, it may suggest that colleges need to provide more 
additional supports beyond the first year to help students continue to succeed. This is consistent with prior research showing that, despite 
the presence of numerous policies to improve access and equity, these policies may not lead to sustainable progress because they fail to 
follow through the lifespan of higher education (e.g., Harper et al., 2009). 


Our study also sheds new light on outcomes for Indigenous students, a population often excluded from higher education research and 
policy discussions (Shotton et al., 2013). In contrast to the results for Black and Hispanic students, we found that the reform had slightly 
greater positive effects for Indigenous students (relative to White students) among the first-year outcomes, and these differential gains 
continued to increase in subsequent years. These differential impacts were substantive in magnitude, with 3.5 credits attempted and 3.1 
credits earned in Year 3 beyond the gains experienced by White students following the reform. This suggests that initial momentum gains 
in the first year may have set some students on a more successful long-term trajectory. It is important to continue to explore how to ensure 
that more students have these types of experiences that position them for long-term success. Research has suggested that change 
initiatives in higher education need to recognize the unique cultural capital that diverse students, such as Indigenous students, bring, and 
use these assets to increase outcomes such as retention (Guillory, 2009). This may include fostering social supports such as multicultural 
offices and student associations, culturally sensitive academic counseling, and support for students to maintain an active presence in their 
home communities. 


Students of multiracial or multiethnic backgrounds appeared not to have gained as much as other student groups in the outcomes 
examined in this study. Although we cannot provide definitive explanations in the current study, it could be the case that students in the 
multiracial or multiethnic group received less targeted support or advising as compared with other groups on campus because of their 
unique backgrounds and the limit of institutional capacity (Renn, 2004; Zook, 2019). These types of constraints may be particularly 
relevant in a large-scale policy change implemented in a short period, such as the developmental education reform in Florida. 


We also explored whether the reform had differential impacts by students’ prior academic preparation, and we found larger effects for 
students on a basic track of high school math or English relative to students on a standard coursetaking track. Similar to our findings by 
demographic subgroups, we found that students on the basic high school track have a greater likelihood of assignment to developmental 
education, and also stand to benefit more if developmental education reform effectively improves student outcomes. The results indicate 
that the reform did have larger effects on credits attempted and earned for students on basic math and English tracks relative to the 
standard track. There were also smaller differential impacts for students on the advanced track relative to students on the standard track. 
Importantly, this differs from prior studies of the impact of developmental education by academic performance, such as Boatman and 
Long (2018), which limited the sample to students who scored below college-ready on a placement test. Florida no longer requires a 
placement test for incoming students, so we cannot explore whether results would differ if placement test scores were used instead of high 
school academic preparation. 


It is important to consider that the effects of Florida’s developmental education reform are due to a comprehensive set of reform efforts, 
and we cannot attribute differences in student outcomes to any individual components of the reform. Simply removing the requirement for 
developmental education is unlikely to result in improvements in student outcomes without providing an extensive network of student 
support services. In a survey of FCS institutional leaders, respondents identified enhanced student advising and support services as among 
the most effective changes resulting from the reform (Hu et al., 2018). This suggests that instead of searching for a single silver bullet to 
improve outcomes for academically underprepared students, policy makers should think about how to create synergy among 
complementary reform activities. This finding is consistent with prior research on other reform efforts, such as City University of New 


York’s Accelerated Study in Associate Programs (ASAP), which emphasize the importance of comprehensive postsecondary 
interventions in promoting student success (e.g., Scrivener et al., 2015). 


Additionally, Florida’s developmental education reform provided colleges with flexibility in implementation. Colleges had a choice of 
selecting one or more instructional modalities for developmental education courses from a list of four options. They also received 
instruction to develop their own plans for making changes to advising and student support services. This left important implementation 
decisions to the expertise of educators and administrators, who would determine which methods might work best for their unique student 
populations. Many college leaders also reported that as the reform progressed, they made changes to instruction, support, and 
infrastructure in response to reviewing data on student outcomes (Hu et al., 2018). Some of these changes seemed quite substantial, such 
as dropping an existing modality for developmental education courses that did not seem to be working well. This suggests that policy 
makers should think of reform efforts as an ongoing learning process informed by the expertise of those responsible for implementation, 
rather than inflexible mandates. 
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Notes 


1. Advanced math courses include probability and statistics, trigonometry, advanced algebra, math analysis, math studies, precalculus, 
calculus, dual enrollment, and any Advanced Placement or International Baccalaureate math courses. 


2. During the time frame of our analysis, students were automatically assigned to a remedial “intensive reading” course if they scored on 
the lower two levels (on a 5-point scale) on the state reading assessment in Grade 10 (see 


References 

Attewell, P., Heil, S., & Reisel, L. (2012). What is academic momentum? And does it matter? Educational Evaluation and Policy 
Analysis, 34(1), 27-44. 

Attewell, P., & Monaghan, D. (2016). How many credits should an undergraduate take? Research in Higher Education, 57(6), 682-713. 


Bahr, P. R. (2010). Preparing the underprepared: An analysis of racial disparities in postsecondary mathematics remediation. The Journal 
of Higher Education, 81(2), 209-237. 


Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in developmental education sequences in community 
colleges. Economics of Education Review, 2(29), 255-270. 


Barhoum, S. (2018). Increasing student success: Structural recommendations for community colleges. Journal of Developmental 
Education, 41(3), 18-25. 


Belfield, C., Jenkins, D., & Fink, J. (2019). Early momentum metrics: Leading indicators for community college improvement. Columbia 


momentum-metrics-leading-indicators.html 


Bers, T., & Younger, D. (2014). The first-year experience in community colleges. New Directions for Institutional Research, 2013(160), 
77-93. 


Bettinger, E. P., Boatman, A., & Long, B. T. (2013). Student supports: Developmental education and other academic programs. The 
Future of Children, 23(1), 93-115. 


Boatman, A., & Long, B. T. (2018). Does remediation work for all students? How the effects of postsecondary remedial and 
developmental courses vary by level of academic preparation. Educational Evaluation and Policy Analysis, 40(1), 29-58. 


Brayboy, B. M. J., Fann, A. J., Castagno, A. E., & Solyom, J. A. (2012). Postsecondary education for American Indian and Alaska 
Natives: Higher education for nation building and self-determination: ASHE Higher Education Report 37. Wiley. 


Chen, G. (2019, October 15). Post-recession cliff looms for community colleges. Community College Review. 
https://www.communitycollegereview.com/blog/post-recession-cliff-looms-for-community-colleges 


Chen, X. (2016). Remedial coursetaking at U.S. public 2- and 4-year institutions: Scope, experiences, and outcomes (NCES 2016-405). 
U.S. Department of Education, National Center for Education Statistics. http://nces.ed.gov.proxy.lib.fsu.edu/pubsearch 


Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlational analysis for the behavioral sciences (2nd ed.). Erlbaum. 


Colyar, J. E., & Stich, A. E. (2011). Discourses of remediation: Low-income students and academic identities. American Behavioral 
Scientist, 55(2), 121-141. 


Deil-Amen, R., & Rosenbaum, J. (2002). The unintended consequences of stigma-free remediation. Sociology of Education, 75(3), 249— 
268. 


Denley, T. (2015). Co-requisite remediation pilot study—fall 2014 and spring 2015. Office of the Vice Chancellor for Academic Affairs, 
Tennessee Board of Regents. http://www.ticua.org.moses.com/meetings resources/sm_files/TBR%20CoRequisite%20Remediation.pdf 


Fish, J., & Syed, M. (2018). Native Americans in higher education: An ecological systems perspective. Journal of College Student 
Development, 59(4), 387-403. 


Flannery, M. E. (2014). The wrong answer on remediation. NEA Higher Education Advocate, 31(5), 3-5. 


Flores, S. M., & Oseguera, L. (2013). Public policy and higher education attainment in a twenty-first-century racial demography: 
Examining research from early childhood to the labor market. In Higher education: Handbook of theory and research (pp. 513-560). 
Springer. 


Florida College System. (2017). The fact book 2017. Florida Department of Education. 
http://www.fldoe.org/core/fileparse.php/15267/urlt/FACTBOOK2017.xls 


Gaertner, M. N., Kim, J., DesJardins, S. L., & McClarty, K. L. (2014). Preparing students for college and careers: The causal role of 
algebra II. Research in Higher Education, 55(2), 143-165. 


Gallard, A. J., Albritton, F., & Morgan, M. W. (2010). A comprehensive cost/benefit model: Developmental student success impact. 
Journal of Developmental Education, 34(1), 10-25. 


Guillory, R. M. (2009). American Indian/Alaska Native college student retention strategies. Journal of Developmental Education, 33(2), 
14-16, 18, 20, 22—23, 40. 


Harper, S. R., & Hurtado, S. (2007). Nine themes in campus racial climates and implications for institutional transformation. New 
Directions for Student Services, 2007(120), 7-24. 


Harper, S. R., Patton, L. D., & Wooden, O. S. (2009). Access and equity for African American students in higher education: A critical 
race historical analysis of policy efforts. The Journal of Higher Education, 80(4), 389-414. 


Hodara, M., & Jaggars, S. S. (2014). An examination of the impact of accelerating community college students’ progression through 
developmental education. The Journal of Higher Education, 85(2), 246-276. 


Hu, S., Mokher, C., Harris, J., Park, T., & Bertrand Jones, T. (2017). Continuing adaptations: Administrators’ perceptions of the third 
year of developmental education reform in the Florida College System. Center for Postsecondary Success. 
http://purl.flvc.org/fsu/fd/FSU_libsubv1_ scholarship submission_1497407239_ 867d0fcd 


Hu, S., Mokher, C., Spencer, H., Hu, X., Park, T., & Bertrand Jones, T. (2018). Exploring institutional change: Administrator's 
perceptions of the fourth year of developmental education reform in the Florida College System. Center for Postsecondary Success. 
http://diginole. lib. fsu.edu.proxy. lib. fsu.edu/islandora/object/fsu%3A5748 14 


Hu, S., Park, T. J., Woods, C. S., Tandberg, D. A., Richard, K., & Hankerson, D. (2016). Investigating developmental and college-level 
course enrollment and passing before and after Florida’s developmental education reform (REL 2017-203). U.S. Department of 
Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational 
Laboratory Southeast. http://ies.ed.gov.proxy. lib.fsu.edu/ncee/edlabs 


Hu, S., Tandberg, D. A., Park, T., Nix, A., Collins, R., & Hankerson, D. (2014). Florida developmental education reform: What do the 
Florida College System institutions plan to do? Center for Postsecondary Success. 


Jaggars, S. S., & Bickerstaff, S.B. (2018). Developmental education: The evolution of research and reform. In M. B. Paulsen (Ed.), 
Higher education: Handbook of theory and research (pp. 469-503). Springer. 


Jaggars, S. S., Edgecombe, N., & Stacey, G. W. (2014). What we know about accelerated developmental education. Columbia University, 
Teachers College, Community College Research Center. https://ccrce-tc-columbia-edu.proxy.lib.fsu.edu/publications/designing- 
meaningful-developmental-reform.html 


Jepsen, C., Troske, K., & Coomes, P. (2014). The labor-market returns to community college degrees, diplomas, and certificates. Journal 
of Labor Economics, 32(1), 95-121. 


Kalamkarian, H. S., Raufman, J., & Edgecombe, N. (2015, May). Statewide developmental education reform: Early implementation in 
Virginia and North Carolina. Community College Research Center. 


Kim, J., Kim, J., DesJardins, S. L., & McCall, B. P. (2015). Completing algebra II in high school: Does it increase college access and 
success? The Journal of Higher Education, 86(4), 628-662. 


Kosiewicz, H., & Ngo, F. (2019). Giving community college students choice: The impact of self-placement in math courses. American 
Educational Research Journal. Advance online publication. https://doi-org.proxy.lib.fsu.edu/10.3102/000283 1219872500 


Leeds, D. M., & Mokher, C.G. (2019). Improving indicators of college readiness: Methods for optimally placing students into multiple 
levels of postsecondary coursework. Educational Evaluation and Policy Analysis. Advance online publication. https://doi- 
org.proxy.lib.fsu.edu/10.3 102%2F01623737 19885648 


Long, M. C., Iatarola, P., & Conger, D. (2009). Explaining gaps in readiness for college-level math: The role of high school courses. 
Education Finance and Policy, 4(1), 1-33. 


Losak, J. (1972). Do remedial programs really work? The Personnel and Guidance Journal, 50(5), 383-386. 


Mokher, C. G., Park, T. J., & Hu, S. (2019, March). The cost implications of developmental education reform: Evidence from students and 
institutions in Florida. Paper presented at the spring research conference of the Society for Research on Educational Effectiveness, 
Washington, DC, United States. 


Mueller, J. A., & Broido, E. M. (2012). Historical context: Who we were is part of who we are. In J. Arminto, V. Torres, & R.L. Pope 
(Eds.), Why arent we there yet? Taking personal responsibility for creating an inclusive campus (pp. 57—103). Stylus. 


Ngo, F., & Melguizo, T. (2015). How can placement policy improve math remediation outcomes? Evidence from experimentation in 
community colleges. Educational Evaluation and Policy Analysis, 38(1), 171-196. 


Okimoto, H., & Heck, R. (2015). Examining the impact of redesigned developmental math courses in community colleges. Community 
College Journal of Research and Practice, 39(7), 633-646. 


Park, T., Woods, C., Hu, S., Bertrand Jones, T., Cig, O., & Tandberg, D. (2018). Gateway course accessibility and the racial/ethnic 
achievement gap: The case of student success following Florida’s developmental education reform. Teachers College Record, 120(12), 1— 
24. 


Park-Gaghan, T. J., Mokher, C. G., & Hu, S. (2019, November 7-9). Do all students benefit from developmental education reform? 
Heterogeneous treatment effects by high school academic preparation. Paper presented at the fall research conference of the Association 
for Public Policy Analysis and Management, Denver, CO, United States. 


Ran, F. X., & Lin, Y. (2019). The effects of corequisite remediation: Evidence from a statewide reform in Tennessee. Columbia University, 
Teachers College, Community College Research Center. https://files-eric-ed-gov.proxy. lib. fsu.edu/fulltext/ED600570.pdf 


Reardon, S., & Stuart, E. (2017). Editor’s introduction: Theme issue on variation in treatment effects. Journal of Research on Educational 
Effectiveness, 10(4), 671-674. 


Renn, K. (2004). Mixed race students in college: The ecology of race, identity, and community on campus. SUNY Press. 
Scott-Clayton, J. (2018). Evidence-based reforms in college remediation are gaining steam—and so far living up to the hype. Brookings 
Institution. https://www.brookings.edu/research/evidence-based-reforms-in-college-remediation-are-gaining-steam-and-so-far-living-up- 


to-the-hype/ 


Scott-Clayton, J., Crosta, P. M., & Belfield, C. R. (2014). Improving the targeting of treatment: Evidence from college remediation. 
Educational Evaluation and Policy Analysis, 36(3), 371-393. 


Scrivener, S., & Weiss, M. J. (2009). More guidance, better results. MDRC. Retrieved from 
http://dev.mdrc.org/sites/default/files/More%20Guidance%20ES_1.pdf 


Scrivener, S., Weiss, M. J., Ratledge, A., Rudd, T., Sommo, C., & Fresques, H. (2015). Doubling graduation rates: Three-year effects of 
CUNY’ Accelerated Study in Associate Programs (ASAP) for developmental education students. MDRC. 
https://www.mdrc.org/publication/doubling-graduation-rates 


Shotton, H., Lowe, S., & Waterman, S. (2013). Beyond the asterisk: Understanding Native American college students. Stylus. 


Spillane, J. (2004). Standard deviations: How local schools misunderstand policy. Harvard University Press. 


Squires, J., & Boatman, A. (2014, June). Essay says achievement gaps are often left out of remedial reform discussions. Inside Higher Ed. 
https://www-insidehighered-com. proxy.lib. fsu.edu/views/20 14/06/02/essay-says-achievement-gaps-are-often-left-out-remedial-reform- 
discussions 


Tampke, D. R. (2013). Developing, implementing, and assessing an early alert system. Journal of College Student Retention: Research, 
Theory & Practice, 14(4), 523-532. 


Tierney, W., & Duncheon, J. (2013, November). Ending remediation is the wrong answer. Jnside Higher Ed. https://www-insidehighered- 
com.proxy. lib. fsu.edu/views/2013/11/08/ending-remediation-wrong-answer-essay 


Valentine, J. C., Konstantopoulos, S., & Goldrick-Rab, S. (2017). What happens to students placed into developmental education? A 
meta-analysis of regression discontinuity studies. Review of Educational Research, 87(4), 806-833. 


Visher, M. G., Butcher, K. F., & Cerna, O. S. (2010). Guiding developmental math students to campus services: An impact evaluation of 
the Beacon program at South Texas College. MDRC. https://www.mdrc.org/publication/guiding-developmental-math-students-campus- 
services 


Zook, K. B. (2019, September). Universities are still struggling to provide for mixed-race students. ZORA. 
https://zora.medium.com/universities-are-still-struggling-to-provide-for-mixed-race-students-d29 1 d89c5b60 


Cite This Article as: Teachers College Record Volume 122 Number 12, 2020, p. - 
https://www-tcrecord-org.proxy.lib. fsu.edu/library ID Number: 23512, Date Accessed: 12/1/2020 12:46:04 PM 


